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Description 

[0001] The present invention relates generally to input procedures and to electronic devices with a plurality of possible 
input parameters. More specifically, the present invention relates to user input prediction, which helps to improve the 
5 usability of complex electronic devices with a minimum of required user input. The present invention relates to user 
specific devices or user unspecific devices with improved usability and minimised necessary user input. The present 
invention relates to a method to improve interface design of electronic devices, too. 

[0002] Actual complex devices or device clusters with a multitude of possible states or features such as e.g. navi- 
gation systems in automobiles require a lot of input until a special navigation can be executed. An internet access point 
10 with a multitude of possible inputs is usually used according to preferred access pattern, e.g. people tend to check 
their e-mail first and then join their preferred chat-rooms or browse for certain keywords in the net. Another example 
is the tendency that people develop certain behaviour patterns in which certain combinations of actuated devices and/ 
or certain coincidences are recurring. 

[0003] The presence of devices or device clusters with a nearly unlimited potential of inputs, features or feature 
15 combinations, in which a single user may forget how a certain definite feature can be accessed -because it is too rarely 

used- shows the need for an input assistance. Especially in systems with a "soft" interface such as computer programs, 

video recorders with on screen display, cars, navigation systems and the like, the tendency to develop a behaviour 

pattern enhances input problems, especially if the user leaves his normal input path, or forgets a certain input and gets 

lost in a multitude of alternatives trying to get back to the known path. 
20 [0004] In order to prevent that people have to use a lot of time to search for a certain input, and to prevent that people 

have to waste too many thoughts on items they might forget, there is a need for a device and a method for input 

prediction in an electronic device. 

[0005] The present invention uses the tendency of people to develop certain behaviour patterns. People tend to 
develop these behaviour patterns nearly in every environment, they are living in. Therefore, the present invention is 

25 applicable to nearly every device people are using. 

[0006] The object of the present invention is therefore to provide a method and related device for context dependent 
user input prediction which offers to the user the most probable next input on the basis of previous input patterns. 
[0007] This is achieved according to one aspect of the present invention by a method for context-dependent user 
input prediction for at least one device in the environment of at least one user comprising the steps of: detecting the 

30 parameters of the states of at least one device, comparing said detected parameters with previously saved parameters, 
obtaining a most probable input proposal according to the result of the comparison, and offering the input proposal to 
at least one user. 

[0008] By comparing a present state with previously saved states the system can help a user or a user support 
program to train a user to execute the desired input, to achieve a certain result. E.g. a safety system in a vehicle can 
35 offer to turn on the lights if the windshield wipers were actuated for more than 1 minute. In this embodiment of the 

present invention, the pre-saved parameters and the comparing algorithm may be updated regularly. 

[0009] Preferably, the method further comprises an identifying step to identify one or more users. By identifying the 

user, the access of the system can be limited. 

[0010] Advantageously, the parameters of the state are handled as vectors. By using vectors, mathematics can be 
40 used to describe and process the parameters of the state of a device in a simple and fast way. The vectors are state 
vectors. The system which is operated by a user is described by a state before the user has made any input and makes 
a transition to a new state after the input is finished. State vectors have multiple dimensions, with different ranges in 
the value of the single parameters. The vector can comprise time and location related parameters, and other state 
related parameters. The system can compare the different parameters by calculating the difference vectors between 
45 the actual vector and the saved vectors. The vector with the shortest difference vector is the most similar vector. The 
difference vector is directly related to an input required to reach the state related to the most similar state vector 
[001 1] Conveniently, the user input prediction method further comprises the step of changing parameters according 
to the reaction to said input proposal or other input. By changing parameters according to the input, the system reacts 
by changing the state of the device by virtue of a single yes/no input. With this, a single unspecific input replaces 
50 multiple specific inputs normally required to change the state of a device. With this, the system updates itself continually 
and supports the user actively by reducing the number of possible and probable states. 

[0012] Preferably, the input prediction method further comprises the step of saving the detected parameters of said 
at least one device. By saving the states of the device, the system can store the behaviour pattern of the user or the 
state pattern of the devices. This leads to a feed back of past behaviour to present behaviour. This method offers a 
55 user to find similar or related situations. In this stage the system recalls previous and similar situations. 

[0013] Advantageously, the parameters include environmental data. By detecting circumstances in which a certain 
input is made, the input proposal to change the state is getting more precise. The state vector comprises more param- 
eters, so the number of similar state vectors in a state is reduced thereby decreasing the number of possible inputs to 
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be offered. Environmental data can be related to the position, tinne, temperature, to weather conditions, or the state of 
devices in the environment of the device. Other parameters can be e.g. the overall power consumption, to detect 
malfunctions or other data lil<e if a certain device with a characteristic power consumption is activated. Other environ- 
mental data can be related to temperature, sunshine, pollution, toxicity, data of vision- of radar- IR- systems, user 
5 related data, e.g. temperature and heart frequency, especially the changes in the parameters. Even the Fourier-trans- 
formations of one or more parameters can be used as parameters. 

[001 4] Conveniently, the input prediction method further comprises the step of weighing said saved parameters. By 
weighing the parameters, the feedback loop is closed, enabling the system to decide if an offer should be made, or 
has been rejected previously. E.g. a current state vector x without user input, i.e. the pure context, is compared with 

10 all stored state vectors X/to find similarities. If similar state vectors X/ exist, the most similar state vector x,- (with its user 
input) is suggested to the user via a user interface as the most probable input. The user can then accept this choice, 
thereby increasing its score and probability for future suggestions, ignore it or reject the choice, thereby decreasing 
its score for future suggestions. The state of the system is described by a vector Xy = (ay, a„gj^/,p/,f/,f5y,U/,/',)"^, where a,- 
denotes the vector of active applications before user input, a„g^/ is the (possibly new) application started or affected 

15 by the user input, p/ is the 3D position of the device when the user starts input, tj is the start time of user input, t^j is 
the start-up time of the terminal before the input can be given, Uy denotes the sequence of user input events (e.g. key 
strokes, voice input or gestures) and fj represents the frequency of use for the state vector Xy. So Xy describes the state 
before user input and the input itself leading to a new state plus how often this specific state vector has been reused 
by the user. 

20 The vector a,- of active applications or the state of one ore more devices and environmental data before user input 
starts is given as ay= (a^ ,a2,...,ayy,0,...0)^, where all possible applications are enumerated starting with 1 and the active 
applications are listed. The number of active applications is given as /Vand the maximum number of active applications 
is limited to N^. So the vector ay has the dimension N^. The N active applications can be sorted in ascending order: 
< ag <. . < a/y. The application started or modified by user input is denoted by B^^^f, again as a application identification 

25 like for ay, 

[001 5] The position vector Pj describes the spatial context in which the user input Uj is made. It is a three-dimensional 
vector with the position data where the user started an input i. The resolution of Pj can be adjusted according to the 
device capabilities: In case of a GPS receiver the resolution can be on the order of meters, for cellular positioning it 
might be 100 m or the diameter of a cell in the wireless infrastructure. A position vector is a useful feature in an mobile 

30 environment, but the invention can be applied to static devices e.g, in a household too. 

[0016] The time of user input fy as well as the system start-up time t^, (last boot time of the device) is measured in 
minutes instead of seconds or on an even more fine-grained time scale since user input does not occur with exact 
timing. If the input prediction system is used in a multi-device environment, the number of time parameters can be 
increased to be able to easily detect hidden input correlation. 

35 [0017] The user input Uycan be stored as a sequence of input events Uy = (u^,U2,...,Uf^,0,...0)'^. How the input is 
actually executed, if the user presses keys, says a voice command or shows a gesture is not important, The system 
may even accept non-human user input e.g. an animal communication and feeding system. The number of user input 
events can be limited to A/^^ events: M < N^j. 

The last entry /y describes how frequently a state vector X/is recalled for input proposals. It ranges between 0 (no recall 
40 at all) and 1 (recalled regularly). The last entry /y can be a vector, e.g. to enable the system to store the behaviour 
patterns of a multiple users. 

[0018] In a multi-user environment a single input pattern prediction system may accept input from multiple users at 
the same time, so the exact fonn of the vector or the vectors to be used is not fixed. The user input vector can be 
coupled to an onginator, or the system may process the data for each user separately. 
45 [0019] The system can propose to enter input information Uy instead of the user input if the similarity or distance of 
the current system state x to any of the stored system states Xy before previous input happened can be quantified. If 
this is solved the system can continuously compare the current state with all the stored states. If the distances are 
small enough, the system can propose the most similar state vector and the affected / activated application is shown 
to the user as the most likely input. One definition of distance measures is straightforward: 

50 

The differences between the current application state a and the situations a,- before user input u,- is given as c/a(a, 
a/) = ||a-aj|. 

The position distance between the current position p and the position py where user input U/was made is defined 
as the Euclidean distance dp{p,p) = ||p-pj|. 

55 

[0020] The recurrence of input patterns Uy is described with two different distance measures: First, the user input 
occurs during a time span after switching on the device. This means that the current time t is compared with the boot 
time fgy. This time span should be the same as the time lag of input Uy after boot time t^f. df^(t,t^,ti,t^j) = \\(t- Q - (tj - tJ 
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II, where time is given in units of e.g. nninutes. 

[0021] Second, the user input is assumed to occur every day at about the same time: There the distance measure 
is defined as diJ<^,t^ =\\^ ' ||mod(24>i^60), i.e. only time differences within one day (= 24*60 minutes) are taken into 
account. This daily repetition pattern can be generalised for other time i ntervals (weel<ly, monthly, yearly recurrence, ...). 
But most likely not enough input statistics will be available for longer time intervals. 

[0022] The input prediction system continuously compares the current state vector x with stored state vectors Xj. 
If one or more state vectors X/ fulfil all distance conditions, 

d^(a,aD < A dp(p,p,) <epA (rffe(f,f^f,,g) < e^^ v c/^^(f,f,.) < e^) 

then the closest state transition / is proposed to the user: 

/ = arg min (d^ +dp+ d^) \f i, where c/f(f,f^ f,, y = min (dt^(t,t^ ti,tj, c/^(f,?/)). 

If one or more state vectors X/ fulfil only two distance conditions 

(G^a <^a^^p< ^p) V (^a <^a^^t< ^t) ^ (dp < Bp A d^ < e^) 
then the closest state transition / is proposed as 



/ = arg min 



d„ + d, if <s^A,d,< s,) 
d^+d^ ifidp <€^A.d,<e,) 



If one or more state vectors x,- fulfil only one distance condition: 



then the state transition / is defined as 



/ = arg min 



dp if dp < €^ 
dg if dj < €f 



[0023] If the system cannot propose any of the stored state vectors X; the distance thresholds e^, Zp and can be 
adjusted in an adaptation process: If no state vector is selected over a longer period of time, the distance thresholds 
are increased in time (without exceeding upper predefined limits). If too many state vectors are similar to the current 
state vector, those thresholds are reduced in time (without falling below minimal thresholds). 
[0024] The system described so far is running automatically in the background. It consists of two operating modes: 
[0025] First, recording of new state vectors X; is performed when user input U/ occurs. In addition the deletion of 
obsolete state vectors is done. Whenever a new state vector x,- is created, the pre-set fj = 1 is made. This way a new 
entry is preferred over older entries with reduced /'y value (i.e. not used for recall regularly). The state vector with the 
lowest f/ value is substituted by the new one if the buffer of possible state vectors is full. 

[0026] Second, comparison of the current state vector x with the state vectors x,- is done. If the user accepts a system- 
proposed input u,- with state vector Xy, the corresponding value is increased as If := a fj^^ -a with 0 < a < 1 
At the same time the other entries X/are reduced as f - a with 0 < a < 1 If the user rejects a system -proposed input 
U; with state vector X/, the corresponding value is decreased as := a // with 0 < a < 1 . 
At the same time the other entries x,- are increased as /) := a -a with 0 < a < 1 . 

[0027] The input proposal from this agent software can be presented to the user as a highly context-sensitive soft- 
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key, maybe as a separate key apart from the rest of the user interface elements. At the same time a graphical icon 
could display which application a^^^^y would be started or affected when the user would accept the input proposal. In 
addition to the affected application the input prediction system could map the distance measure for the input proposal 
to a graphics element like a magic eye known from old radios in order to express the reliability of the proposal. 
5 [0028] Once the user selects the input proposal x^-the input prediction system reproduces the user input in order 
to automate the input. 

[0029] Ideally, operating frequently used applications and features reduces to one key to be pressed by the user, i. 
e. pressing the accept button for the input prediction system proposal. The system will work the better the more fre- 
quently a feature is used. 

10 [0030] In the case that specific algorithms are used, pairs of parameters which seem to be correlated can be coupled 
together, so that if one parameter reaches a predetermined level, the other parameter is offered to be changed ac- 
cordingly. 

[0031] It is possible that a user changes parameters by personal input, like coupling two parameters by connecting 
it by direct input or via e-mail or the like. 
15 Together with the above mentioned user identification system, the system can adapt to different behaviour patterns of 
different users, The system may even recognise a certain user on characteristic initial input patterns, or characteristic 
environmental data. 

Changes in the state of a device can occur without an input of a user, having the consequence of a user input. The 
system can then develop a certain behaviour pattern to avoid a certain user input automatically. 

20 [0032] Preferably, the offered change in the state is executed automatically. By executing an input automatically, the 
system just informs the user of a change in the state of a device. The user can accept an input by not denying an offer. 
This decreases the number of necessary inputs to a minimum. This feature can be activated by a specific input, or by 
a predetermined value of the weight of the proposed input. The system can offer different responses to a certain offer. 
In the simplest case, a "one button"- solution, the user accepts the offer, or the offer was not executed. Another one 

25 button solution is "Yes" if the button is actuated long and "No" if the button is actuated for a short period of time. Two 
button solutions may comprise a "Yes"- and a "No"-key, The interface may comprise several other different input options 
like "Never ask again", "remind me later", "remind me earlier", "undo", "redo", " execute automatically" and so on. 
[0033] Advantageously, the input prediction method further comprises the steps of transmitting and/or receiving data. 
By transmitting and receiving data, a system according to the present invention can access data, like an external time 

30 signal or position signal from a positioning system such as GLONAS (Global Navigation System) or GPS (Global 
Positioning System). The data can be exchanged via radio, telematic service (Short IVIessage System), e-mail, IR- 
connections Bluetooth, a wired connection. By using data transfer the system can be distributed between a mobile 
input device and a remote service unit. The service unit can be a service computer being connected to a personal input 
device via radio, and to different input devices via a wired connection. 

35 [0034] Conveniently, the input prediction method further comprises the step of compressing the data. By compressing 
the data, the amount of stored state vectors, or the amount of stored parameters can be increased, of the time intervals 
between the detection of states of a device can be decreased, increasing the dynamic of the input prediction system. 
[0035] According to another aspect of the present invention the object of the present invention is achieved by a 
computer program comprising program code means for perfomning the steps, describes in the embodiments of the 

40 method for the input prediction method, when the program is run on a computer or a mobile terminal device. By using 
the program on a computer, a user behaviour pattern can be used to prevent that a user have to restart the com 
[0036] According to yet another aspect of the present invention, a computer program product is provided that com- 
prises program code means stored on a computer readable medium for perfonriing the embodiments of the above 
mentioned user input prediction method, when said program product is run on a computer, a personal interface device, 

45 a mobile terminal device and so on. 

[0037] According to yet another additional aspect of the present invention an electronic device capable of context- 
dependent user input prediction is provided, which comprises: means for storing parameters, means for detecting the 
parameters of the state of at least one device, means for comparing said detected parameters of said at least one 
device with previously saved parameters, means for offering changes of the parameters of said at least one device, 

50 and means for user input. By means for storing parameters, the present invention is capable of being trainable, to 
certain behaviour patterns, of a certain user, but the system can be pre-trained too. Means for detecting parameters 
is a necessary feature, to be able to detect the state and the circumstances of an input, a device or a device cluster. 
[0038] Preferably, the device further comprises means for identifying said at least one user. Equipped with a user 
identification mans, the input predictions can predict behaviour patterns of different users, the user identification means 

55 can be used for selective access, e.g. that children can not access selected devices or selected inputs. The user 
identification can be used to pay for certain services resulting from an input. The user identification can be a SIIVI- 
(Subscriber Identification Means) card together wit a PIN number, or other personal identification means. The input 
device can be incorporated in a mobile phone, a palmtop computer or the like, which are usually protected against 
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unauthorised access. 

[0039] Advantageously, the device comprises means for transforming said parameters of the state of said at least 
one device into state vectors. The use of state vectors makes it possible to use well known mathematical rules to 
describe and process the state related data of devices. The algorithms for processing the data can be used independ- 
5 ently from the actual device, the structure of the input pattern. For the mathematical algorithm ist is not important if the 
parameter describes a position in time, space, a temperature, a loudness, a power consumption, flow rates or any 
other setting of a device, or state of an environment. 

[0040] Conveniently, the device further comprises means for changing parameters according to an answer to said 
input proposal or other input. As described above in the description of the method, the answer to the proposed input 

10 is the input which is required to train the system. 

[0041] Preferably, the device further comprises means for transmitting data. Advantageously the device further com- 
prises means for receiving data. This enables the device to access data that are normally not accessible for a single 
device. The exchange of data enables the system to be distributed. With data exchange an interface device can com- 
municate with a service computer or like, to remote control the input prediction system. The data exchange can be 

15 used to train e.g. a Radio or a TV set to reduce volume if e.g. the telephone rings. The user input prediction system 
can be incorporated in a single device or be used to remote control several devices. The user input prediction system 
can be offered by a service provider to remote manage certain inputs, even if the user is actually not present. The 
system can e.g. offer to close windows, if sensors detect precipitation, or heavy weather conditions. 
[0042] Advantageously the device further comprising input means and displaying means. These features are less 

20 important for systems with the main dedication to search for behaviour patterns, to improve e.g. the quality of a certain 
product. Input and display means can be separated in a single external device like a personal interface device or so. 
Input and display means can be incorporated in the environment of the user. For example a car with an input prediction 
system can be fitted with a display element, and input elements at the steering wheel or the gear change lever. In a 
simpler version, the standard input elements of the car can be fitted with e.g. light emitting diodes, which flash, if the 

25 input prediction system uses offers an input. 

[0043] Conveniently, the device is connected to a network. With the connection to a network the system can access 
data, which are normally not accessible, as weather forecasts, times of sunrise and sunset and so on. 
[0044] In a preferred embodiment, the device, further comprises means for changing the state of the device auto- 
matically. A perfect trained user input prediction system need no special user input, cause every possible user input 

30 is executed automatically at the best suited moment. Such a system would, if it was not turned off, e.g. during vacation 
open the windows turn on the lights, the Radio the TV, even if nobody is actually at home, repelling burglars, because 
the house seems to be inhabited. 

[0045] According to another aspect of the invention an interface device for the use in a context-dependent user input 
prediction system is provided which comprises: means for transmitting and receiving data, means for displaying the 
35 received data, user input means. The interface device can be a universal input element connected to, or built in into 

at least on other device. An interface device may be integrated in a wrist watch. 

[0046] Preferably the interface device further comprises means for storing data and means for calculating. With the 
ability to store and process data, the interface device can offer all the advantages of a input prediction system, and 
can be incorporated as a module into other devices to enable a input prediction feature. 

40 [0047] According to another aspect of the present invention, a server in a network is provided, which comprises 
means for storing data, means for receiving data, means for comparing said received data with previously saved data, 
means for generating an input proposal, means for transferring data, and means for changing data. A server in a 
network can offer complete input prediction even with low cost equipment. If the system only transfers state related 
data, if the state has changed, and only which parameter of the present state has changed, the amount of data to be 

45 transferred can be minimised. 

[0048] Preferably, the data transferred is related to the state of devices remote from the server. With such a system 
input prediction can be offered with devices with relatively low calculation power, and the ability to be remote controlled 
via a network. 

[0049] The behaviour patterns of a user can be used to improve the input speed on these input patterns, but they 
50 can be used to improve the input itself. In some computer programs a user can combine several inputs to input chains, 
where the computer only need one input to execute the whole chain. The merit of this invention is that the system itself 
can offer the user to execute multiple inputs, if he is used to do so. The system can be able to offer the creation of 
input chains, if the TBI (Time between inputs) is below a determined value. 

[0050] As people are starting to deal with behaviour patterns, even an inverted use of input patterns can be used to 
55 improve technical devices, The usability of the method or the idea are not limited at all. E.g. Inverted behaviour pattern 
input can be used in simulators to improve the actual accuracy of training in a simulator, to prevent a standard behaviour 
pattern in case of an undefined emergency. E.g. a pilot's training program should not follow the preferences of a certain 
examiner. 
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[0051] An additional aspect of paramount importance resides in a service not only putting up personal behaviour 
data of a large amount of persons leading to an information base wherein new devices may be designed, which are 
optimised in design, durability, and especially in possible input paths. Imagine people have enough money to spend it 
for an input prediction system, and they send you their personal behaviour data. The amount of data, even low-level 
5 motion data that can be derived from location data of mobile phones, being deleted without deriving cumulative be- 
haviour patterns, is a waste of data resources. The data that are usually provided by polling institutes and inquiries, 
only because people not yet recognised the value of big data streams. Data streams can help to determine the actual 
spread, and the actual use of the devices, connected to the input prediction system. 

[0052] In the following, the invention will be described in detail by the way of non limiting examples referring to the 
10 enclosed drawings in which: 

Figure 1 is an embodiment of a user input prediction system, which is implemented in a device with limited calcu- 
lation power. 

15 Figure 2 is an embodiment of a user input prediction system with a spread devices in the environment of a user. 

Figure 3 is an embodiment of the present invention, wherein said user input prediction system is implemented in 
a vehicle. 

20 [0053] Figure 1 describes an embodiment of the present invention, in which the present invention is adapted to be 
used with a low weight, low cost terminal device, with low calculation and processing power. The present example 
describes a mobile phone, which transfers position data and other user specific data via WAP 3 (Wireless Application 
Protocol ) to a WAP/internet gateway 3. The data is then transferred via TCP/IP (Transfer control protocol/Internet 
Protocol) to a service provider 5 connected to a network 4. The service provider detects position and other data of the 

25 terminal device according to input patterns of the user, and saves them in user and device specific memory devices 
6,7,8. If the system only stores and transmits changes in the state of the device 2 or better only stores the parameters 
of the state before and after an input the amount, of data to be transferred and stored is minimised. The service provider 
5 tracks the user in a virtual model according to the received data. Time only changes in the state of the device don't 
need to be transmitted as world wide time signals are available. 

30 [0054] With such an architecture the terminal device only needs very low processing power to provide the advantages 
of the present invention. The terminal device can offer the user to call a certain number, when leaving work, if the user 
used to call this number automatically. A distributed system is not limited in the amount of data, that can be stored or 
processed. Therefore a distributed input system can be used even to identify long-temn behaviour pattems. If the 
terminal device includes a WAP browser, the system can even store and predict a browsing pattern. A WAP application 

35 would simplify the WAP browsing process, and reduce the number of necessary input. With a reduced input frequency, 
and reduced input variables such as Yes/No, the user can use a trained terminal device with only few keys to be 
pressed, making big keyboard obsolete. So the number of high quality input keys can be decreased. 
[0055] Figure 2 shows the system as it can be configured in a distributed environment with distributed single devices. 
The present embodiment of the invention is an input prediction system for a multitude of distributed devices which are 

40 connected to a service computer 9. The service computer is connected to several memory devices 6, 7, 8, to store all 
available parameter of every connected device in the environment of a user 1 . The devices actually depicted are a 
heating 30, a lamp 31, a TV (television) set 32, a coffee machine 33, a VCR (video recorder) 35, a PC (personal 
computer) 34, a radio 36, an alarm clock 37 and a access system 37. 

[0056] Standard conditions are that the user alarm clock 37 rings at 6 o'clock in the morning, the radio 36 was 
45 activated at 6.05, the light 31 was turned on at 6. 10, the coffee machine 33 at 6. 15. At latest at 7 o'clock in the morning, 
all activated devices where turned off, and the access system recognises that the user leaves home. At 5.55 o'clock 
p.m. the access system 39 recognises the return of the user, at 6.00 the heating 30 was turned on, at 6.15the PC 34 
was turned on, 6.1 6 e-mail was checked, 6.20 radio 36 was turned on. At 8.00 p.m. the TV set 32 vas turned on 8.01 
the radio 36 was turned off, at 8.15, the TV channel was changed several times, than certain video text pages where 
50 read back, than the TV set 32 was turned off. At 9.15 the PC 34 was turned off , and the TV set 32 was turned on again 
on a certain channel, and was turned off at 9.55 again. At 1 0.00 p.m. the heating 30 was turned off, the radio 36 was 
turned on till 10.30, and the light 31 was turned off 10.45. 

[0057] In the present example, all the devices are connected to, and can be remote controlled by the service computer 
9. In a first stage, the service computer 9 watches the behaviour of the user according to a feedback from the connected 
55 devices. During this stage, the user can control the devices directly, or via a personal interface device 10. Preferably, 
the personal interface device 10 is connected to the service computer 9 via a wireless connection. The personal inter- 
face device 1 0 may be integrated in a cordless telephone or in mobile phone. The interface device 1 0 needs at least 
two keys and a display. The system collects every information and stores it. The service computer 9 can store the 
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information e.g. as vectors, containing at least one time scale and tlie states of tlie different devices. Tlie service 
computer 9 recognises that, if tlie user leaves the environment, all the devices where turned off. 
[0058] The state of the every device can be stored as a state vector wherein every parameter of the vector is related 
to a setting of the device. For example the state of the lamp 31 has one parameter, which can be one of two values 0 

5 and 1 . The heating 30 has only one state, one parameter, but the parameter can comprise every value between 0 and 
maximum power. The state of the radio 36 has much more parameters: on/off, stereo/mono, frequency, loudness and 
so on. So the state of the radio 36 can be described in a state vector with at least four different parameters, wherein 
two of them has two discrete values, and the other two can be every value between a maximum and a minimum. 
[0059] The vectors are state vectors and define different spaces with different numbers of dimensions. E.g. the lamp 

10 31 only defines a 1 -dimensional space, with only two different values, if this vector is combined with other different 
vectors, like the time a simple graph like that of a digital signal can be drawn. The values of the heating 30 cover a 
certain positive range, limited by the zero and the maximum line. The heating 30 and the lamp together with the time 
define a 3-dimensional space. This space can be drawn in two parallel graphs, and if the states of these two devices 
are saved for every day, a behaviour pattern may get visible, that e.g. the in the user is used to turn on the light 31 

15 every moming from 6.00 to 7.00 and in the evening from B.OO to 10.00, while the heating 30 was usually turned on at 
the evening 9.00 to 10.00. So the system can detect that, if the heating 30 was turned on, the light 31 was always 
turned on too, so that the system can offer the user to turn on the light 31 if he had turned on the heating 30. The 
values of the state of the two devices can be comprised in one 2-D vector. 

[0060] If the vector comprises the time as an additional variable the vector comprises a 3-D space. In a 3-D space, 

20 this sounds not very exciting, but in a multidimensional space with a multitude of devices, the user can benefit from 
the fact that the system can save time, even in situations usually not even regarded. The main benefit of the user input 
prediction system is that it is searching autonomously for behaviour patterns, to reduce the overall input. The other 
advantage is, that the system can adapt to different situations and autonomously integrate additional devices without 
loosing the data referring to the standard behaviour. 

25 [0061] A new device connected to the user input prediction system changes the dimensions of the behaviour space, 
with an additional device as the coffee machine 33, the TV set 32, the Personal Computer 34, and the like, the state 
vector is a high-dimensional vector in a high-dimensional space. A user will not fill this high dimensional state space 
uniformly, but will tend to develop certain input patterns, which are related to each other or to other parameters. A set 
of parameters can be related to different time scales. Different time scales increase the conditioning speed of the 

30 system, and help to discover long term behaviour patterns. 

[0062] Every device connected to the system may comprise a single time scale, started when the device was turned 
on, a time scale may be restarted every day, and one time scale may be restarted every week. Longer times between 
single time restarts, like every month may be useful e.g. for money related transactions. Yearly restarts can be useful, 
but have the major drawback, that the time since the system can recognise certain patterns is too long. Estimate, that 

35 the system recognises patterns after 1 0 periods, without bothering the user too much, the system would need 1 0 years 
to operate satisfyingly. Ten years of operation time is longer than the average operation time of an average device, 
Additionally the amount of data accumulating during the term of 1 to 1 0 years is much greater compared to the expected 
benefit. But, in the future, storage space surly wont be a problem. 

[0063] The depicted environment typically describes a household environment with typical household devices. The 
40 interface device 10 connects directly to a service computer, which remote controls all the devices in the household. 
The interface device 1 0 can be incorporated e.g. in a mobile phone, and therefore can be moved to different environ- 
ments like to work, to a car, or to other frequently visited environments. Such an embodiment could be operated in with 
a service provider in a network as depicted in the figure 1 , or with distributed service computers in distributed environ- 
ments. The user may be identified by a personal identification means such as a SIM (Subscriber Identifications Means) 
45 or other devices. A mobile Interface device 1 0 ensures that a service computer can contact the user, even if the user 
already left the environment which may require an input. This would enable the service computer e.g. to offer an absent 
user to turn off an oven. 

[0064] If the interface device 1 0 is incorporated in a mobile telephone or other portable device, the user need not to 

learn new input methods or structures from different input devices. This seems to be no problem, as two buttons and 
50 a display won't be difficultto use, but the user input interface may consist of keys, voice recognition, gesture recognition 
and other human-originated input recognition means. The same difficulty is to be expected with the display element. 
The display can be a optical, acoustical or structural. 

[0065] Figure 3 describes an embodiment of the present invention that is built into a vehicle. Compared to the situation 
in Figure 2, the of the vehicle or the states of devices onboard the vehicle is related to the actual position of the user 
55 and/or the vehicle. Therefore the parameters used for the input prediction comprise the actual position of the Vehicle. 
Even for automotive application a 3-D position vector is useful, because the actual 3-D position can indicate that the 
vehicle is actually in a tunnel, which causes the driver to behave different compared to on surface conditions. The 
rising number of electric or electronic applications replacing standard mechanical applications, open the possibility to 
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control these applications electronically. In the present example, a user has two primary destinies: work 56 and friends 
52. 

[0066] Thefirst idea of thesystem is, thatthe system detects the behaviour of the userto derive an behaviour pattern, 
to offer an input the user may have forgot. If the system has been trained well enough, the user can fully control his 
5 preferred personal settings of all applications of the vehicle according to his personal preferences with only a few input 
elements. The system can detect, at which time of the day the user usually turns the lights on, is listening to a certain 
radio station if he is driving to work 56, and to another if he is going to visit his friends 52. The system can automatically 
deactivate a navigation system if the user is entering the car at a time, the user usually uses to drive to work 56 or visit 
his friends 52. 

10 [0067] The system can include external sensors to detect weather conditions, and related behaviour of the driver. 

The system can detect, that the user is used to open the windows near his home 54, and closes it, if he reaches the 
motorway always, if the external temperature is higher than 15°C. The system can offer special ventilation settings, 
under wet and cold conditions, even before the user starts the engine to prevent the steaming of the windows. Or the 
vehicle can offerto start de-icing of the windows automatically, if the user used to do so under similar weatherconditlons. 

15 [0068] The system may identify a certain user by certain sensor input or by certain behaviour patterns like if the user 
is used to put on the security belt first and then start the engine, or the like. After being trained, the system can offer 
the driver to open and close the windows automatically. The vehicle adapts automatically to the behaviour patterns of 
the driver. The system may even adapt to a refuel behaviour, if the user prefers a certain fuel station, a certain fuel 
level, the cheapest station to refuel. In a first step the system asks the driver if a certain input is desired or not. This 

20 Question can easily be answered with Yes/No. The system may include other possible answers like "ask me later", 
"Ask me earlier", "never ask me again" or "execute always automatically". 

[0069] The system may include pre-saved, preferred behaviour patterns, to increase the safety and comfort of all 
people. In the case of a vehicle application that includes for example, that if the user actuates the windscreen wipers, 
the system offers to actuate the rear window wiper, and to turn on the lights, even if the driver is not used to do so. 

25 The system can limit the running conditions of the engine, to reduce wear during warm up of the motor. In the future, 
thesystem can be used to train drivers. For example, a the behaviour pattern data can help an assurances to calculate 
the premium of a certain dhver. Additionally, a highly pre-saved system, that can not be turned off, can help the user 
to get lower assurance premiums. A high-resolution input prediction system always includes a kind of a "drive recorder" 
that can help to analyse the condition in which an accident occurred. A high-resolution input prediction system can 

30 help to sell the car, cause the saved data can prove that the motor has not been ran under overspeed conditions. 

[0070] It should be noted that state describes a state of a device, the state of the device can be on/off, related to the 
position, to features and settings, or even to a program which is run on the device and the like, wherein the device can 
be a single device such as a computer, or multiple devices in the environment of a user, which the user can access, 
like all the devices that can be found in a household or in a car and so on. 

35 [0071] This application contains the description of implementations and embodiments of the present Invention with 
the help of examples. It will be appreciated by a person skilled in the art that the present invention is not restricted to 
details of the embodiments presented above, and that the invention can also be implemented in another form without 
deviating from the characteristics of the invention. The embodiments presented above should be considered illustrative, 
but not restricting. Thus the possibilities of implementing and using the invention are only restricted by the enclosed 

40 claims. Consequently various options of implementing the invention as determined by the claims, including equivalent 
implementations, also belong to the scope of the invention, 

[0072] When searching the most similar past behaviour vector by comparing the actual parameters of the actual 
state with previously saved parameters the system is looking for identity in most parameters which corresponds to 

non-weighing the parameters. The most similar state of the past is chosen, 
45 However, if the system detects that certain applications always coincide with certain other parameters, such as time 
or location, it may additionally emphasize or weigh these parameters in away that with lacking the coinciding param- 
eters in the actual state there is never similarity to past states. 

[0073] This means that certain combinations of parameters acquire a higher local probability resulting in different 
assessment. 

50 

Claims 

1 . IVIethod for context-dependent user input prediction for at least one device in the environment of at least one user 
55 comprising the steps of: 

detecting the parameters of the actual states of said at least one device, 

comparing said detected parameters with previously saved parameters of said at least one device. 
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obtaining a most probable next input proposal according to tlie result of tlie comparison, 
offering said input proposal to at least one user. 

2. Method according to claim 1 , further comprising the step of identifying said at least one user. 

3. Method according to claim 1 or 2, wherein said parameters of the state of said at least one device are handled as 

vectors. 

4. Method according to any of the preceding claims, further comprising the step of changing parameters according 
to the answer of the user to said input proposal or other input. 

5. Method according to one of the preceding claims, further comprising the step of saving said detected parameters 
of said at least one device. 

6. Method according to any one of the preceding claims, wherein said parameters include environmental data. 

7. Method according to any of the preceding claims, further comprising the step of weighing said saved parameters, 

8. Method according to any one of the preceding claims, wherein said offered input to the user is executed automat- 
ically. 

9. Computer program comprising program code means for performing all the steps of any one of the preceding claims, 
when said program is run on a computer or a mobile terminal device. 

1 0. Computer program product comprising program code means stored on a computer readable medium for performing 
the method of any one of claims 1 to 8 when said program product is run on a computer or a mobile terminal device. 

11. Electronic device capable of context-dependent user input prediction comprising: 

means for storing parameters, 

means for detecting the parameters of the state of at least one device, 

means for comparing said detected parameters of said at least one device with previously saved parameters, 
means for offering an input proposal regarding said at least one device on the basis of the comparison, and 
means for user input. 

12. Device according to claim 11 , further comprising means for identifying said at least one user. 

13. Device according to claim 12, further comprising means for transforming said parameters of the state of said at 
least one device into state vectors. 

14. Device according to any of claims 11 to 13, further comprising means for changing parameters according to an 
answer of the user to said input proposal or other input. 

15. Device according to any one of the claims 11-14, further comprising means for transmitting data. 

16. Device according to any of claims 11 - 15, further comprising input means and displaying means. 

17. Device according to any of the claims 11 - 1 6, wherein said device is connected to a network. 

1 8. Device according to any of claims 11 - 1 7, further comprising means for changing the state of the device automat- 
ically. 
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Fig. 2 
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